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E xtended learning opportunities (ELOs) provide safe, 
structured learning environments for students out- 
side the traditional school day. ELOs include afterschool 
and summer learning programs as well as before-school, 
evening, and weekend programs.^ ELOs come in many 
forms and can include tutoring, volunteering, academic 
support, community service, organized sports, home- 
work help, and art and music programs. ELOs are based 
in schools, 2 1 st Century Community Learning Centers, 
child care centers, and community-based organizations, 
such as 4-H Clubs and Boys and Girls Clubs. No matter 
where they are located, ELOs complement what children 
and youth learn during school in ways that support stu- 
dent success. For this reason, effective ELOs should be 
considered an integral part of state elementary and sec- 
ondary (K-12) education systems. 

Research demonstrates that high quality ELOs can 
improve a variety of student outcomes. Participation 
in high quality ELOs is linked to improvements in aca- 
demic achievement, school attendance, student engage- 
ment, work-study habits, and social and emotional 
development.^ In addition, ELOs offer support for work- 
ing families and can help foster stronger links among 
schools, families, and communities.^ The supports and 
services that high quality ELOs provide are particularly 
important for low-income and minority youth who 
often lack sustained access to enriching activities and 
academic support during nonschool hours.^ 

All ELOs, however, do not produce similar results. In 
fact, low quality ELOs fail to show positive impacts and 
can even have negative effects on children. ^ Therefore, 
governors, chief state school officers, and other state 
leaders should act to support the development, sustain- 
ability, and availability of high quality ELOs. 

High quality ELOs share a number of common features.^ 
High quality ELOs set focused and challenging goals for 
their participants that are tied to clear programmatic 
missions, and they use frequent evaluation to improve 
their programs over time. In addition, high quality ELOs 
provide an array of content-rich programming that en- 
gages participants and builds their academic and nonaca- 
demic skills. Staff members in high quality ELOs 
develop positive, constructive relationships with partici- 



pants. Furthermore, LEO program coordinators and 
staff members develop strong partnerships with school 
administrators and teachers, participant family members, 
and community organizations. Importantly, high quality 
ELOs recruit, train, support, and retain qualified and 
effective program staff. Finally, high quality ELOs have a 
low participant-to-staff ratio, an appropriate total enroll- 
ment, sufficient program resources, and the ability to 
sustain funding over the long term. 

The importance of ensuring LEO quality has grown in 
recent years as state investments in ELOs have skyrock- 
eted. In fiscal year 2006, states budgeted at least $897 
million for afterschool programs; this conservative esti- 
mate excludes funding for sports and tutoring programs 
as well as most non-dedicated funding streams in which 
afterschool programs were just one of a number of eligi- 
ble expenses.^ States also administer federal LEO funding 
streams. These include the 21st Century Community 
Learning Centers (21st CCLC) program, which at $1.1 
billion for fiscal 2008 is the largest dedicated source 
of federal funding for ELOs, and the Child Care and 
Development Fund (CCDF), a non-dedicated funding 
stream with estimated federal expenditures of $ 1 billion 
in fiscal 2005 for school-age youth attending child care 
centers.^ Both chief state school officers, as heads of the 
agencies that administer the bulk of ELO dollars, and 
governors, as chief executives of their states, have strong 
incentives to focus on ELO quality. Better quality ELOs 
mean improved student outcomes and higher returns on 
investment. 

To improve ELO quality, state leaders have initiated efforts 
to develop program standards, create program self-assess- 
ment tools, and provide technical assistance to local pro- 
grams. State leaders can build on and strengthen these 
efforts by integrating them into a broader state ELO qual- 
ity system. State leaders can take the following actions to 
implement a comprehensive state ELO quality system: 

►► Establish an ELO quality team of key stakeholders 
to envision, develop, and administer a state ELO 
quality system. Because public funding for ELOs 
flows from federal, state, and local entities via educa- 
tion, human services, and other funding streams, sup- 
porting ELO quality must be a team effort. Statewide 
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afterschool networks, now present in three-quarters of the 
states, offer one example of an effective vehicle to collabo- 
rate across sectors, agencies, and levels of government.^ 
These public-private coalitions bring 
together state leaders and grassroots supporters to pursue 
state goals through ELOs, and they have been at the fore- 
front of developing and coordinating policies 
to improve ELO quality. 

►► Identify federal and state funding sources to support 
ELO quality. Set-asides in federal ELO funding streams 
provide a base states can use to finance their quality 
efforts. Both the federal Child Care and Development 
Fund and the 21st Century Community Learning Cen- 
ters program have a percentage of their funds set aside 
for states to use to improve program quality. States can 
also incorporate quality set-asides into state ELO fund- 
ing streams. For example, the California Department of 
Education can set aside 1.5 percent of funds appropri- 
ated for the After School Education and Safety Program 
to provide technical assistance, evaluation, and training 
services and to offer local assistance funds to support 
program improvement.^^ 

►► Specify state goals for ELOs and set research-based 
ELO program standards. Defining state goals for 
ELOs helps states determine what high quality ELOs 
should look like, how to support them, and how to 
hold them accountable. ELO program standards artic- 
ulate the conditions that, if met, will enable programs 
to succeed in meeting their goals. Michigan, for exam- 
ple, has created Model Standards for Out-of-School 
Time Programs to provide a flexible model of ELO 
excellence. States should base their goal- and standard- 
setting on emerging research that links specific ELO 
program features to improved participant outcomes. 

►► Measure the extent to which ELOs meet program 
standards and demonstrate expected results. For 
example, Massachusetts evaluates ELO effectiveness 
with its Survey of Afterschool Youth Outcomes 
(SAYO). The state also helps ELOs compare their prac- 
tices to program standards with its Assessing After- 
school Program Practices Tool (APT). SAYO and APT 
meet criteria for effective evaluation; they are adapt- 
able, rely on multiple sources of data, are statistically 



sound, and are reasonable and research-based. Measures 
such as these enable states to determine whether pro- 
grams are meeting policymakers’ expectations and to 
support ELO programs as they strive for excellence. 

►► Provide incentives to improve ELO quality. Increas- 
ingly, states are spurring improvements in ELO effec- 
tiveness with quality rating systems and mechanisms 
that tie funding to program quality. Quality rating 
systems evaluate ELOs according to whether they 
meet specific benchmarks in a public and easy-to- 
understand manner (e.g., using a scale of one to five 
stars) . Examples of mechanisms to tie funding to 
ELO quality levels include competitive grant pro- 
grams, in which only the highest quality programs 
receive funding, and tiered reimbursement systems, 
which provide more funds to afterschool programs 
with higher levels of quality. 

►► Support a strong ELO workforce. ELO quality rests 
largely on the strength of the individuals working in the 
programs. Because of this, states can boost ELO quality 
by supporting the recruitment, retention, and develop- 
ment of effective ELO staff. To create a framework for 
professional development activities, states can establish 
core knowledge and competencies that delineate what 
ELO workers should know and be able to do. For exam- 
ple, the Kansas and Missouri Core Competencies for 
Youth Development Professionals form the basis for 
ongoing training activities. 

►► Connect students with high quality ELOs. To ensure 
that students have access to ELOs, states can perform 
ELO supply and demand analyses, target resources to 
underserved populations, and inform parents and stu- 
dents of high quality ELOs. Many states have under- 
taken a supply and demand analysis in recent years. 
Montana, for example, has generated county-by- 
county supply and demand maps that show where pro- 
grams are available and what percentage of school-age 
children with working parents is served. 

Implementing these seven strategies can help state lead- 
ers boost ELO quality to better support children, fami- 
lies, and communities. 
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ELOs Important? 





I. Why Are High Quality ELOs Important? 



E xtended learning opportunities (ELOs) support state 
goals in education, economic development, and other 
arenas by providing safe, structured learning environ- 
ments for students outside the regular school day. ELOs 
include afterschool and summer learning programs as 
well as before-school, evening, and weekend programs. 
ELOs vary significantly in goals, content, structure, and 
duration. However, they typically offer programming that 
includes academic support (e.g., acceleration, remedia- 
tion, and individualized tutoring) along with some com- 
bination of enrichment, mentoring, sports, or other 
extracurricular activities . ^ ^ 

ELOs are critical supports within high-functioning educa- 
tion systems. Because ELOs offer small, flexible learning 
environments, they often provide an ideal context for 
addressing the specific educational and developmental 
needs of individual students. Student access to ELOs has 
grown rapidly during the past decade, as evidenced by 
increases in federal, state, and local funding. These invest- 
ments have been fueled, in part, by research demonstrating 
that ELOs can produce these outcomes: 

►► Boost students’ academic gains. ELOs can improve 
student achievement in reading, mathematics, and 
other subjects. For example, the Study of Promising 
Afterschool Programs compared elementary school stu- 
dents who participated in high quality afterschool pro- 
grams for two years with students who spent one to 
three days per week without adult supervision after 
school. The average math score increase for program 
participants was higher than that of 70 percent of low- 
supervision students. Moreover, a review of LEO 
evaluations found that programs focused on improv- 
ing reading or mathematics achievement for at-risk 
students were generally successful. Because of their 
impact on student achievement, ELOs can be a key 
strategy to narrow the achievement gap (see How Can 
High Quality ELOs Narrow the Achievement Gap?). 

►► Increase student engagement. A review of 27 experi- 
mental and quasi-experimental program evaluations 
found that ELOs have been linked to better attitudes 
toward school, higher educational aspirations, and 
improved school attendance. For example, an evalu- 



ation of the Los Angeles Better Educated Students for 
Tomorrow afterschool program found that 83 percent 
of students liked school more since participating in 
the program, with over half indicating that they liked 
school “a lot more.”^^ 

►► Cultivate students’ work-study habits. ELOs can posi- 
tively impact participants work-study habits. For exam- 
ple, the Study of Promising Afterschool Programs found 
that 58 percent of elementary school students who regu- 
larly attend high quality ELOs outgained the average 
unsupervised student with respect to teacher-reported 
work habits over the course of one year.^^ Similarly, in 
contrast to their peers in unsupervised settings, elemen- 
tary school students who regularly attend high quality 
LEO programs show increases in persisting with tasks, 
paying attention in class, following directions, complet- 
ing classroom assignments on time, and completing 
homework.^® 

►► Improve student behavior and social and emotional 
development. Students say participation in ELOs 
helps them feel safe, maintain self-control, curtail fight- 
ing, and avoid premarital pregnancy and risk-taking 
behaviors such as alcohol and drug use. Participation in 
ELOs also has been linked to decreases in suspensions 
and expulsions, improved self-confidence and self- 
esteem, and increased positive interactions with peers 
and adults.^^ For example, a meta-analysis of ELOs 
using research-based instructional methods found that, 
relative to the average student in a control group, the 
average student in these ELOs scored 14 percentiles 
higher on measures of self-perception, 1 0 percentiles 
higher on measures of school-bonding, 12 percentiles 
higher on measures of positive social behaviors, and 9 
percentiles better on measures of drug use.^^ 

►► Support working families. Parents whose children 
attend ELOs worry less about their childrens safety 
and are better able to balance work and family life.^^ 
Moreover, parents say their child’s participation in 
ELOs has enabled them to find better jobs, work 
longer hours, and miss less work.^^ For example, in an 
evaluation of ELOs in New York City, 94 percent of 
parents rated the programs as convenient, 60 percent 
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missed less work because of the programs, 59 percent re- 
ported that the programs helped them keep their job, 
and 54 percent said the programs helped them work 
more hours.^^ 

►► Build stronger connections among families, schools, 
and communities. Parents of children who regularly 
attend ELOs that seek to engage families are more likely 
to be involved in school-related events, attend parent- 
teacher conferences, and help students with their home- 
work. For example, an evaluation of the federal 21st 
Century Community Learning Centers program found 
that parents of middle school participants were more 



likely to attend parent-teacher organization meetings 
(34 versus 28 percent), to attend afterschool events 
(47 versus 40 percent), and to volunteer to help out at 
school (18 versus 15 percent) In addition, families 
with students in ELOs have credited these programs 
with increasing their awareness of community sup- 
ports. Particularly for low-income parents who may 
not have experienced educational success in their own 
schooling, ELO staff can play a critical role, serving as 
liaisons to teachers and administrators and empowering 
parents as advocates for their children. 



How Can High Quality ELOs Narrow the Achievement Gap? 



The term “achievement gap” is used to denote ongoing dispari- 
ties in educational achievement between low-income and more 
affluent students and between black and Hispanic students and 
their white and Asian peers. Achievement gaps are evident as 
early as kindergarten, and they often persist and widen as stu- 
dents advance in their schooling. Consider these statistics: 

►► Among children entering kindergarten in 1998, cognitive 
achievement scores of those with high socioeconomic status 
were, on average, 60 percent above the scores of children 
with low socioeconomic status. Moreover, mathematics 
achievement was 2 1 percent lower among black children 
and 1 9 percent lower among Hispanic children than it was 
among white children, on average. 

►► On the 2007 National Assessment of Educational Progress 
(NAEP) reading assessment, 14 percent of black fourth 
graders and 17 percent of Hispanic fourth graders scored 
proficient, compared with 42 percent of white fourth graders 
and 45 percent of Asian fourth graders. In eighth grade, 12 
percent of black students and 14 percent of Hispanic stu- 
dents scored proficient in reading, compared with 38 percent 
of white students and 40 percent of Asian students.^^ 

►► On the 2007 NAEP mathematics assessment, 15 percent of 
black fourth graders and 22 percent of Hispanic fourth 
graders scored at or above proficient in mathematics, com- 
pared with 51 percent of white students and 59 percent of 



Asian students. In eighth grade, 1 1 percent of black students 
and 15 percent of Hispanic students scored proficient in 
mathematics, compared with 41 percent of white students 
and 49 percent of Asian students.^^ 

Achievement gaps also feed gaps in educational attainment: 

►► In 2003, the graduation rate for white students was 78 per- 
cent, compared with 72 percent for Asian students, 55 per- 
cent for black students, and 53 percent for Hispanic 
students.^^ 

►► As of 2005, among individuals ages 16 to 24, 6 percent of 
whites were high school dropouts, compared with 1 0 per- 
cent of blacks and 22 percent of Hispanics.^^ 

►► In 2003, just 10 percent of Hispanic and 18 percent of African 
American young adults earned a bachelor s or higher degree, 
compared with 34 percent of white students. 

Gaps in achievement and educational attainment are attributa- 
ble to disparities in supports, resources, experiences, and op- 
portunities that students encounter both in school and out of 
school. Within schools, low-income and minority students are 
less likely to receive instruction from highly qualified teachers, 
attend schools with adequate resources, and have access to a 
rigorous and challenging curriculum.^^ 
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Although many ELOs have demonstrated positive im- 
pacts such as these, too many ELOs fail to reach their 
potential. This is one of the reasons why research find- 
ings on ELOs’ effects have been mixed. Although high 
quality ELOs have demonstrated positive impacts on 
youth outcomes, low quality ELOs fail to show positive 
impacts on student outcomes and can even have negative 
effects on children.^^ 

Although high quality ELOs have 
demonstrated positive impacts on youth 
outcomes, low quality ELOs fail to show 
positive impacts on student outcomes and 
can even have negative effects on children. 



For instance, a case study of a program that exhibited 
relatively negative staff-child interactions, poor emo- 
tional climate, and few provisions for child autonomy 
found that attendees showed declines in their report card 
grades, peer relations, and work habits. 

Because of the divergent results of high and low quality 
ELOs, state leaders face a quality imperative. Fortu- 
nately, chief state school officers and governors have 
administrative, budgetary, and legislative levers at their 
disposal to achieve the promise of ELOs. 



Differences outside the school walls, however, also contribute sig- 
nificantly to the achievement gap. This can be seen most clearly 
by examining seasonal differences in learning. When school is 
out over the summer months, low-income children experience 
an average loss in reading achievement that approximates two 
months of schooling while middle- and upper-income children 
tend to hold steady or make slight advances. These losses, 
which result in great part from unequal access to learning oppor- 
tunities in childrens home and community environments, accu- 
mulate over students’ academic careers. A longitudinal study of 
students attending Baltimore City public schools found that 66 
percent of the ninth-grade achievement gap between low-income 
and upper-income students was due to differences in summer 
learning during the elementary school years.^® 

High quality ELOs can help narrow the achievement gap by 
boosting student achievement for low-income and minority stu- 
dents. As described above, the Study of Promising Afterschool Pro- 
grams found increases in mathematics achievement among 
elementary students attending high quality afterschool pro- 
grams. Of the elementary students sampled for that study, 89 
percent received free or reduced-price lunch at school and 88 
percent were students of color. High quality summer programs 
have also proven effective for at risk students. A study of the 
Building Educated Leaders for Life (BELL) summer program, 
designed to improve low-income children’s academic skills, 
found that participants gained about one month’s worth of read- 
ing skills more than their counterparts in the control group. 



In fact, several studies have found that students at risk of poor 
academic performance often achieve the greatest academic gains 
when participating in ELOs.^^ For example, an evaluation of 
ELOs in New York City found that the average first-year math 
score increase for poor students actively participating in ELOs 
was higher than that of 57 percent of similar students who did 
not participate. In contrast, this statistic was 45 percent among 
wealthier students.^^ 

Despite the demonstrated success of high quality ELOs in boost- 
ing academic success for low-income and minority students, par- 
ents of these students report greater difficulty in locating high 
quality ELOs that are affordable, interesting, convenient, age- 
appropriate, and administered by trustworthy adults. This chal- 
lenge is greatest during the summer months, when low-income 
and minority parents are far more likely to be worried about their 
children falling behind academically and about finding affordable 
programs that interest their children.^^ 

By increasing access to high quality ELOs among low-income 
and minority students, state policymakers can compensate for 
unequal access to learning opportunities outside of school and 
narrow the achievement gap. Reform initiatives that focus on the 
regular school day are critical. But to be most effective, school- 
based reforms must be coupled with efforts to provide low-in- 
come and minority students with high quality learning 
opportunities outside the regular school day and year. 
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High Quality ELOs 




G iven the positive relationship between ELO quality 
and program outcomes, understanding what high 
quality ELOs look like is crucial. Fortunately, recent 
research has tied specific ELO features to improved par- 
ticipant outcomes. Much of this research is based on 
programs that serve high percentages of low-income or 
minority students. Shared features of high quality pro- 
grams include:^^ 

►► A clear programmatic mission, focused and challeng- 
ing goals, and frequent evaluation that supports 
ongoing program improvement. To better understand 
the importance of these features, consider the James 
Irvine Foundations eight-year CORAL (Communities 
Organizing Resources to Advance Learning) initiative. 
CORAL provides ELOs to elementary school youth in 
five California cities. It aims to increase the academic 
achievement of youth, but a midterm evaluation found 
that academic results were lacking. To remedy the situa- 
tion, CORAL changed its mission to focus more specifi- 
cally on improving struggling readers’ literacy skills. The 
program also developed two goals aligned with that mis- 
sion: improving participants’ reading levels as measured 
by grades and standardized tests and helping partici- 
pants develop a love of reading. To meet these goals, 
CORAL implemented literacy programming three to 
four days per week for 60 to 90 minutes each day.^^ Fur- 
thermore, leaders trained staff on teaching strategies to 
support student literacy, monitored staff activities, 
coached staff as necessary, and collected and analyzed 
data to track progress.^^ These changes led to increases 
in reading scores. In the most recent evaluation of the 
program, the participants gained, on average, nearly half 
a grade’s worth of reading.^^ 

►► An array of content-rich programming that engages 
participants and builds their academic and nonaca- 
demic skills. High quality ELOs are not glorified 
babysitting. Participants are not engaged in busywork, 
watching television, or just hanging out. After all, the 
“L” in “ELO” stands for learning. Participants in high 
quality ELOs are building skills and knowledge in 
both academic and nonacademic areas. In a high 
school job training program, for example, students 
might learn programming languages for Web design 
and gain an understanding of proper workplace behav- 
ior. For learning to be most successful, participants 



need to be engaged with content that is appropriately 
challenging (i.e., content that is neither too easy nor 
too difficult). To boost participant engagement, high 
quality ELOs give students the opportunity to choose 
activities that will motivate them. For example, pro- 
grams may offer culturally relevant activities to engage 
participants and build their self-confidence. The vehi- 
cles used to deliver content are also critical in inspiring 
participants to learn. Educational approaches in high 
quality ELOs include debate, hands-on activities, one- 
on-one tutoring, and project-based learning. 

►► Positive, constructive relationships between staff and 
participants. High quality programs help participants 
forge strong connections with caring adults. This is par- 
ticularly important for students who may feel isolated, 
disconnected, and unsuccessful during the regular 
school day. In practice, positive staff-child relationships 
are characterized by staff treating program participants 
with acceptance and respect, providing emotional sup- 
port, setting appropriate limits and behavioral expecta- 
tions, and communicating high expectations. Students 
with supportive relationships early in high school are 
twice as likely as the average youth to be doing well, 
overall, with respect to economic security, health, and 
community involvement at the end of high school.^^ 

►► Strong connections with schools, families, and 
communities. These connections include schools and 
ELOs sharing resources and information on student 
goals, ELOs partnering with local universities to 
expand participants’ postsecondary horizons, and 
ELO staff providing structured opportunities for par- 
ent involvement in an open and welcoming environ- 
ment. Connections like these increase the likelihood 
that students will participate in ELOs at a high level 
and boost ELO relevance. 

►► Qualified, well-supported, and stable program staff. 

Whether a result of formal or informal training, educa- 
tion, or individual talents and experience, program 
leaders and frontline staff must be able to support stu- 
dent learning and develop positive relationships. Pro- 
grams working with academically at-risk students 
should emphasize attracting and retaining high quality 
staff members who have experience working with 
at-risk or disadvantaged youth and addressing motiva- 
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tional issues. Some ELOs have found improved student 
outcomes in programs with lower staff turnover, more cer- 
tified teachers, more staff with college degrees, and direc- 
tors with higher levels of education.^^ 

►► A low participant-to-stafF ratio and an appropriate 
total enrollment. Having a sufficient number of qual- 
ified staff relative to the number of participants is 
essential to address individual participant needs, sup- 
port individual or small-group activities, and develop 
positive relationships. The National AfterSchool Asso- 
ciation recommends a maximum participant-to-staff 
ratio of 1 5 to 1 for participants age six and above.^^ It 
is possible, however, that ratios should be even lower: 
A recent study of high quality ELOs found that, for 
most activities, the participant-to-staff ratio fell below 
10 to 1.5^ Moreover, total enrollment should not over- 
whelm program space. 

►► Sufficient program resources and the ability to sus- 
tain funding over the long term. To support partici- 
pant skill development and mastery, high quality 
ELOs have sufficient program resources, including 
safe and adequate space and facilities, age- appropriate 
curricula and materials, and a location that is accessi- 
ble to participants and families. Ongoing financial 
resources are needed to acquire physical resources and 
support a qualified staff Consequently, the abilities to 
attract and use diverse funding and to develop and 
use external partnerships are extremely important. 



Ideally, these research-based promising practices would be 
embodied in all ELOs, but ELOs are too often plagued by 
uneven quality. To address this problem, many states have 
initiated efforts to improve ELO quality. For example, 
states have developed program standards, created program 
self-assessment tools, and provided technical assistance to 
local programs. To be most effective, these efforts should 
be based on research and best practices. For example, state 
ELO program standards should embody the shared fea- 
tures of high quality ELOs just described. Furthermore, 
efforts to improve ELO quality should not stand alone. 
Rather, they should be a part of a comprehensive state 
ELO quality system that functions to ensure and enhance 
ELO effectiveness and efficiency. By spearheading the 
development of a state ELO quality system, governors and 
chief state school officers can substantially boost ELO 
effectiveness and position states to realize the benefits of 
high quality ELOs. 



By spearheading the development of 
a state ELO quality system, governors 
and chief state school officers can 
substantially boost ELO effectiveness 
and position states to realize the 
benefits of high quality ELOs. 
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III. state Actions to Develop an ELO Quality System 




To develop an ELO quality system, state leaders can take these actions: 



1. Create an ELO quality team of key stakeholders 
to envision, develop, and administer an ELO qual- 
ity system. 

2. Identify federal and state funding sources 

to support the state ELO quality system. 

3. Set state ELO goals and program standards that 
are research-based and adaptable to the wide variety 
of programs. 

4. Measure ELO performance by determining the 
extent to which programs meet state ELO program 
standards and demonstrate results tied to state 
goals and program mission. 

5. Provide incentives to improve ELO quality 

through quality rating systems and mechanisms 
that tie funding to program quality. 

6. Support a strong ELO workforce by helping to 
recruit talented individuals into the field and by 
providing for professional development. 



7. Connect students with quality ELOs by perform- 
ing an ELO supply and demand analysis, targeting 
resources to ensure availability and access to high 
quality programs among those with the greatest 
need, and informing parents and students of high 
quality opportunities. 

Importantly, each strategy needs to be integrated and 
coordinated with the others to ensure a comprehensive 
approach to improving program quality. 

States do not need to develop an ELO quality system from 
scratch. Rather, they can build off of existing quality efforts 
related to the child care system, 21st Century Community 
Learning Centers (21st CCLC), and Supplemental Educa- 
tional Services (SES), along with state-funded and private 
programs. Too often, these efforts to improve program 
quality occur in isolation from one another. By creating a 
comprehensive ELO quality system, state policymakers can 
ease provider access to multiple funding streams, reduce 
duplication or confusion of efforts, provide state ELO 
quality leaders the opportunity to learn from each others’ 
efforts, and increase efficiency. 



Action 1: Create an ELO Quality Team 



S tates should build an ELO quality team of key stake- 
holders to craft and guide the implementation and 
management of an ELO quality system. A successful 
ELO quality team will include individuals from different 
sectors, agencies, departments, and levels of government 
who have expertise to bring to bear on ELO quality. The 
governor’s office, state education agency, and child care 
office are key agencies that should take part in statewide 
ELO quality discussions. Other relevant agencies include 
health, labor, youth development, and juvenile justice. 
Cross-agency collaboration counteracts what would oth- 
erwise be a diffuse approach to fostering ELO quality at 
the state level, driven by fragmented funding streams that 
lead to differences in program goals, administrative struc- 
tures, and philosophical approaches. 



As important leaders and communicators in their states, 
chief state school officers and governors are well posi- 
tioned to bridge these differences by convening a team to 
help ensure ELO excellence. For example, Arkansas Gov- 
ernor Mike Beebe created the Governor’s Task Force on 
Best Practices for After-School and Summer Programs. 
The task force included representatives from the state’s 
departments of education and human services, K-12 and 
postsecondary institutions, the private sector, and local 
and state elected government. The task force set forth a 
framework for ELO standards and recommendations for 
promoting quality in its report. Enriching Arkansas Chil- 
drens Lives through high quality Out-Of-School Activities P 
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In building an ELO quality team, states 
can use preexisting collaborative groups, 
such as statewide afterschool networks, 
children’s cabinets, and P-16 councils, to 
take advantage of available state capacity. 



In building an ELO quality team, states can use preexisting 
collaborative groups, such as statewide afterschool net- 
works, childrens cabinets, and P-16 councils, to take 
advantage of available state capacity. For example, an ELO 
quality team can be a working group of a commission or 
task force or the quality committee of a statewide after- 
school network. In Illinois, a legislative task force issued 
recommendations in 2002 on supporting quality ELOs 
statewide. The state s afterschool network — the Illinois 
After-school Partnership — has since been instrumental in 
implementing those recommendations. One of the net- 
works workgroups is creating a common framework to 
evaluate ELO outcomes. Statewide afterschool networks 
like the Illinois After-school Partnership exist in 38 states 
with support from the Charles Stewart Mott Foundation.^^ 
These networks are based on statewide, regional, and local 
partnerships, and they focus on boosting ELO effective- 
ness. A statewide afterschool network can be a key resource 
and perhaps lead the effort to develop a statewide system 
for ensuring high quality ELOs. 

An ELO quality team needs to reach out to additional 
stakeholders to take advantage of their expertise, create buy- 
in, and ensure that state ELO quality policies are workable, 
given realities on the ground. For example, New^brks 21st 
CCLCs must use a quality self-assessment tool to improve 
ELO effectiveness. The New York State Afterschool Net- 
work led a collaborative effort by more than 200 afterschool 
program providers, stakeholders, and advocates to develop 
the tool. The diversity of the coalition that developed the 
tool ultimately strengthened the end product. Similarly, the 
development of ELO quality systems should be informed 
by stakeholders such as students, parents, teachers, princi- 
pals, businesses, foundations, program providers, and com- 
munity-based organizations. Elected offtcials at the local 
and state levels, such as mayors and state legislators, should 
also play a key role. 



Collaboration at the state level that leads to a shared vision of 
what ELO quality is and how to pursue it should drive coor- 
dination within agencies to help implement that vision. 
Within state education agencies, for example, individuals 
who administer Title I, the 21st CCLC program. Supple- 
mental Educational Services (SES), Safe and Drug-Free 
Schools and Communities grants, and other student support 
programs can work together to address ELO quality from 
the state education agency perspective. The North Carolina 
Department of Public Instructions 21st CCLC staff mem- 
bers incorporate consultants from SES, Teacher Quality, 
Reading First, Exceptional Services, and additional divisions 
into their training and orientation sessions for 21st CCLC 
grantees. In the Alabama Department of Education, staff in 
charge of 21st CCLC and the states service learning pro- 
gram cosponsor and convene conferences focused on quality. 
By fostering cooperative efforts such as these, a state s ELO 
quality team can ensure sound implementation of its ELO 
quality system. 

An ELO quality team should be able to evolve as a state 
moves from envisioning a coordinated ELO quality system, 
to implementing the system, and, finally, to managing the 
system. In the long run, states can create more centralized 
governance structures for ELOs. Some models for this con- 
cept currently exist, and other states are considering moves 
in this direction. Connecticut’s legislature has established 
an After School Advisory Council that requires the state’s 
commissioner of education, commissioner of social services, 
and executive director of the children’s commission to work 
with other council members to produce biannual ELO pol- 
icy recommendations. In Rhode Island, leaders from the 
governor’s office, the state education agency, the legislature, 
and the statewide afterschool network, as well as others, 
have been discussing the possibility of creating a state-level 
coordinating body for ELOs to manage ELO funding 
streams from different departments, administer a statewide 
quality system, and integrate ELOs into state education 
reform plans. Newly developed state governance models in 
the field of early childhood education can serve as proto- 
types for ELOs. For example, Pennsylvania’s Office of 
Child Development and Early Learning brings together 
early childhood education programs from both the educa- 
tion and public welfare departments into one office that is 
on both departments’ organizational charts. Washington 
has created its Department of Early Learning — a cabinet- 
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level state agency combining initiatives that had previously 
been under the Department of Social and Health Services, 
the Department of Community Trade and Economic De- 
velopment, and the Office of the Superintendent of Public 
Instruction. 



Regardless of the extent to which a state consolidates gover- 
nance of ELOs, establishing an ELO quality team is a key 
step in developing a state ELO quality system. Establishing 
such a team enables states to reduce duplication, improve 
efficiency, and develop a shared approach to ELO quality. 



Action 2: Identify Funding Sources for ELO Quality 



S tates should identify federal and state funds they can 
use to develop an ELO quality system to ensure that 
children receive the highest possible quality of care, dol- 
lars are spent as efficiently as possible, and ELO efforts 
address state goals. 

States administer federal ELO funding streams, includ- 
ing the 2 1 St Century Community Learning Centers 
(2 1 St CCLC) program, which at $ 1 . 1 billion for fiscal 
2008 is the largest dedicated source of federal funding 
for ELOs, and the Child Care and Development Fund 
(CCDF), a non-dedicated funding stream with esti- 
mated federal expenditures of $1 billion in fiscal 2005 
for school-age youth attending child care centers. 

Other federal funding streams that support ELOs in- 
clude Safe and Drug-Free Schools and Communities 
grants, food and nutrition funds. Title I education fund- 
ing, and Temporary Assistance for Needy Families. Local 
and private investments are also an integral part of the 
ELO funding landscape. 

21st CCLC and CCDF provide all states with a preexist- 
ing funding base to support state ELO quality. Each of 
these programs has a quality set-aside: a percentage of 
funds reserved for improving program quality. Federal 
law requires states to set aside a minimum of 4 percent 
of federal and state CCDF funds to improve child care 
quality and accessibility for low-income families. In 
2004, states invested more than double the required 
amount — $920 million, or 10 percent of total CCDF 
funds — to improve program quality. These investments 
in early childhood and school-age care often focus on 
promoting safe and healthy environments and providing 
professional development.^^ 



State education agencies may designate up to 2 percent 
of 2 1 St CCLC funds for peer review of grant applica- 
tions and supervision of award-making to eligible organ- 
izations and up to 3 percent of funds to:^° 

►► Monitor and evaluate programs and activities; 

►► Provide training, capacity-building, and technical 
assistance to grantees; 

►► Evaluate the effectiveness of programs and activities; and 

►► Offer training and technical assistance to eligible 
grant applicants. 

Some states coordinate CCDF and 21st CCLC quality 
set-asides to support broad-based ELO quality efforts at 
the state level.^^ As a case in point, Missouri uses funding 
from both sources to support the Missouri Afterschool 
Resource Center, which provides training, support serv- 
ices, and technical assistance to afterschool programs in 
the state.^^ By taking a systemic approach, states can 
make dollars invested in ELO quality efforts go further. 

States can also incorporate quality set-asides into state ELO 
funding streams. For example, the California Department 
of Education can set aside L5 percent of funds appropri- 
ated for the After School Education and Safety Program to 
provide technical assistance, evaluation, and training serv- 
ices and to offer local assistance funds to support program 
improvement.^^ The importance of these types of set-asides 
has grown in recent years as state investments in ELOs 
have skyrocketed. Many states fund ELOs through dedi- 
cated funding streams, general funds, or departmental 
budgets. In fiscal year 2006, states budgeted at least $897 
million for afterschool programs; this conservative estimate 
excludes funding for sports and tutoring programs as well 
as most non-dedicated funding streams in which after- 
school programs were just one of a number of eligible 
expenses.^^ 
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Per-participant cost estimates for high quality ELOs vary 
widely, ranging from $449 to $7,160 per year7^ Much of this 
variation can be attributed to differences in program charac- 
teristics, such as scale, location, and staffing patterns, as well 
as to methodological differences in costing studies, including 
sample sizes, how costs are calculated, whether in-kind 
resources are taken into account, and whether startup, operat- 
ing, and system-building costs are included^^ To get a better 
understanding of ELO program funding, consider the follow- 
ing high-profile examples: 

►► The California After School Education and Safety pro- 
gram, funded at $550 million and the largest state invest- 
ment in ELOs in absolute terms, provides programs with 
$7.50 per participant per day for a three-hour per day 
program and requires a 1 to 3 local monetary or in-kind 
match. For a five-day-a-week elementary school program at 
180 days per school year, this translates to $1,800 per year 
($1,350 in state funding and $450 in match per year). 
California raised the state reimbursement rate from $4.90 
to $7.50 per participant per day after policymakers deter- 
mined that a higher reimbursement rate was needed to sup- 
port high quality programs. 

►► Kentucky’s Extended Service School Initiative provides 
every school district in the state with funds to support 
students with academic difficulties. The per-participant 
cost averaged about $2,390 per participant per year and 
ranged from about $1,000 to about $4,220, among 10 
sites studied during the 1999-2000 school year. Ken- 
tucky’s Extended Service School programs offer extra in- 
structional time outside regular school hours and can 
include afterschool or before-school programs, evening 
sessions, Saturday learning opportunities, summer pro- 
grams, and intersession programs. 



►► New York’s Advantage After School Programs receive 
$1,250 per student from state-administered Temporary 
Assistance for Needy Families funding.^^ In addition to this 
funding, which stood at $28.2 million in 2007-08, pro- 
grams receive support from school, community, and pub- 
lic-private partnerships. The initiative aims to provide 
quality youth development opportunities to school-age 
children and youth. Programs operate three hours per day 
for five days per week during the regular school year and 
may elect to operate during school breaks. Programs may 
also extend into the evening hours, particularly if serving 
older youth. 

►► Building Educated Leaders for Life (BELL), a nationally 
recognized afterschool and summer learning program 
staffed with certified teachers and tutors that serves nearly 
12,000 students in 75 public and charter school sites in 
Maryland, Massachusetts, Michigan, and New York, has 
direct costs of $2,000 per student per year for its afterschool 
program and $2,000 per student per year for its six-week 
summer learning program.® BELL’s indirect costs are 7 per- 
cent for administration and 5 percent for fundraising.^^ 



1 
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Better quality ELOs mean improved 
student outcomes and higher returns 
on investment. 



Although funding is a critical component for supporting 
state ELO quality infrastructure, the level of funding in- 
dividual ELOs receive also has a direct impact on the 



quality of the services they can provide (see How Much 
Do ELOs Cost?). Both governors, as chief executives of 
their states, and chief state school officers, as heads of the 
agencies that administer the bulk of ELO dollars, have 
strong incentives to fund ELO quality. Better quality 
ELOs mean improved student outcomes and higher re- 
turns on investment. 



Action 3: Set State ELO Goals and Program Standards 



o build the foundation of an ELO quality system, 
states must define the goals they expect high quality 
ELOs to meet and articulate ELO program standards 
tied to these goals. ELO program standards describe the 
conditions that, if met, will enable programs to succeed 
in meeting state and program goals. ELO program stan- 
dards serve as a model of ELO excellence, providing 
guidance to states on how to invest in and support 
ELOs, to new programs on how to effectively meet their 
goals, and to existing programs on how to address areas 
in need of improvement. 



Too often, states neglect to clearly define 
the outcomes they expect from ELOs. 

Too often, states neglect to clearly define the outcomes 
they expect from ELOs. Without an understanding of 
the challenges a state is seeking to address through 
ELOs, it is difficult to determine what ELOs should 
look like, how to support them, and how to hold them 
accountable. With a shared understanding of state goals 
for ELOs, on the other hand, a state can define a high 
quality ELO as one that effectively and efficiently meets 
state and program goals. 

ELOs have roots in the fields of education, youth devel- 
opment, child care, and prevention. Consequently, states 
typically consider ELOs as a means to: 

►► Support children’s school success as well as their 
broader intellectual, social, physical, emotional, and 
psychological development; 



►► Assist working parents by ensuring that their chil- 
dren have a safe and constructive place to be after 
school or during the summer; and 

►► Keep communities strong by generating school-com- 
munity connections and preventing risky youth behav- 
iors that involve crime, drugs, and sexual activity. 

States face a challenge in developing ELO program stan- 
dards because there is no one-size-fits-all ELO. Different 
ELO options are necessary to meet the varied needs and 
demands of students, families, and communities. Although 
all high quality ELOs will share certain features, other fea- 
tures will differ from program to program. For example, a 
high quality afterschool program focused on hands-on sci- 
ence learning will look different than a high quality sum- 
mer program focused on civic awareness and community 
engagement. Similarly, features of high quality ELOs will 
vary according to participant age. 

Given the great diversity in ELO programs, there are 
three approaches states can take to develop ELO pro- 
gram standards tied to state goals. First, they can create 
standards that are sufficiently broad-based so as to apply 
to most ELO programs. North Carolina’s standards, for 
example, describe a high quality ELO as one that “offers 
activities appropriate to the ages and skill levels of partic- 
ipants” and that “provides a variety of activities that 
reflect program mission.” Second, states can narrow the 
scope of their standards to concentrate on a subset of 
programs. For example, Michigan’s standards focus on 
programs serving students in kindergarten through grade 
eight. Finally, states can take a modular approach, where 
certain standards apply to all programs and other stan- 
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dards vary according to program goals. Massachusetts’ 
Assessing Afterschool Program Practices Tool uses differ- 
ent standards for programs focused on different aca- 
demic subjects. 

ELO program standards should be grounded in research 
on the features of high quality afterschool programs (see II. 
Features of High Quality ELOs). In Massachusetts, the 
Department of Elementary and Secondary Education, the 
National Institute on Out-of-School Time, and the Massa- 
chusetts After-School Research Study collaborated to pro- 
duce standards based on evidence-based ELO best practices 
consistent with desired program outcomes. In developing 
these standards, state partners reviewed research on after- 
school, education, and the arts and gleaned perspectives 
from experts in these fields. By informing the development 
of its standards with information on research-based prac- 
tices, Massachusetts was able to provide programs with a 
roadmap to quality that has proven effective. 

To ease the development of ELO program standards, states 
should consider building on existing state or local efforts. 
For example, Rhode Island has adopted statewide quality 
standards, initially developed by the Providence After- 
school Alliance for local programs. Illinois, has explored 
adapting the goals and objectives of the Illinois Teen 
REACH afterschool program for wider use in the state. 



States can also take advantage of a broad range of ELO 
program standards used beyond their own borders. At least 
42 ELO quality standard documents and assessment tools 
are being used nationwide, according to a scan conducted 
by the Harvard Family Research Project and based, in part, 
on initial tracking undertaken by the Council of Chief 
State School Officers.^^ In Michigan, the state department 
of education, along with representatives from three school 
districts across the state, used diverse references to inform 
the development of the Michigan Model Standards for 
Out-of-School Time Programs, including standards pub- 
lished by the National Association of Elementary School 
Principals, the School-Age Care Environment Rating Scale, 
and the National AfterSchool Associations Standards for 
Quality School-Age Care.^^ 

Although only a handful of states have already developed 
state ELO program standards, a number of others are 
currently on the path to developing their own, including 
Connecticut, Illinois, New Mexico, and South Car- 
olina.^^ This interest in developing state ELO program 
standards reflects the increasing importance state leaders 
see in providing ELOs with a blueprint they can follow 
to meet state goals in support of children, families, and 
communities. 
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Action 4: Measure ELO Performance 



S tates should develop measures of ELO quality to deter- 
mine whether programs are meeting policymaker 
expectations and to support ELO programs in their pursuit 
of excellence. There are two main aspects of ELO quality 
that states should measure: the extent to which programs 
meet state ELO program standards and the degree to 
which programs demonstrate results consistent with state 
and program goals. 

States can develop assessment tools to measure how 
ELOs stack up against state quality standards. Cur- 
rently, most state-developed ELO assessment tools are 
program self-assessments, which help ELOs improve 
their practices on an ongoing basis. For example, the 
New York State Afterschool Network s Quality Self- 
Assessment Tool uses a common set of quality standards 
to help programs assess, plan, design, and execute strate- 
gies for ongoing program improvement.^^ In Massachu- 
setts, the Assessing Afterschool Program Practices Tool 
(APT) helps programs evaluate how their practices com- 
pare with evidence-based best practices that are consis- 
tent with their expected program outcomes. In both 
these states, 21st CCLCs are required to use these tools. 
Other states that have developed self-assessment tools 
include Colorado, Missouri, New Hampshire, which 
adapted the National Community Education Associa- 
tions Continuous Improvement for Afterschool Pro- 
grams tool, and North Carolina. The Michigan 
Department of Education is currently rolling out a cus- 
tomized version of High/Scopes Youth Program Quality 
Assessment tool that is focused on self-assessment and 
tied to the state s ELO program quality standards. 

States could consider using similar tools for monitoring 
or accrediting programs. 



There are two main aspects of ELO 
quality that states should measure: the 
extent to which programs meet state 
ELO program standards and the degree 
to which programs demonstrate results 
consistent with state and program goals. 

Determining how well ELOs embody state program 
standards is critical, but other factors are equally impor- 
tant. States should also seek to measure the effects that 
programs have on participants. Ideally, states would also 
measure the effects that programs have on families and 
communities. Currently, this is an underdeveloped area 
in the realm of state ELO policy because of the difficulty 
of teasing out ELO program impacts from other influ- 
ences. For example, it is difficult to determine how 
much of an increase in a student s test score is due to 
activities in school, in an ELO, or at home. Despite dif- 
ficulties such as these, it is vital to adopt measures of 
program effects so that state governments can more fully 
evaluate the results of their ELO efforts and investments. 
The Massachusetts Department of Elementary and Sec- 
ondary Education, in cooperation with the National In- 
stitute on Out-of-School Time, has taken the lead in 
developing a research-based ELO evaluation system 
called the Survey of Afterschool Youth Outcomes 
(SAYO).^^ SAYO uses brief pre- and post-participation 
surveys to track changes in participants’ outcomes. Like 
Massachusetts’ APT, programs have some choice regard- 
ing what outcomes they choose to measure, depending 
on their mission (see Guiding Principles for Measuring 
ELO Quality: The Massachusetts Approach). 
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States can collect, store, and analyze 
data to inform efforts to improve 
program quality. 



States can collect, store, and analyze data to inform 
efforts to improve program quality. One approach states 
may take involves incorporating data on ELOs into exist- 
ing education data systems. Delaware, for example, has 
developed a longitudinal education data system to track 
student achievement over time. The state recently built a 



Student Service Program Evaluation component into this 
data system to track the participation of individual stu- 
dents in 21st CCLC and SES. By combining student 
data on ELOs with school records and state achievement 
test data, Delaware plans to conduct a sophisticated 
analysis to investigate the impact of ELOs on student 
achievement over time. States may also consider linking 
ELO data to other state databases (e.g., health, juvenile 
justice, and social services). This would allow states to in- 
vestigate the impact of ELOs on nonacademic outcomes, 
including those related to supporting working families 
and keeping children and youth safe and out of trouble. 






Guiding Principles for Measuring ELO Quality: The A/[assachusetts Approach 



A closer examination of SAYO and APT — Massachusetts’ 21st 
CCLC quality measures — illuminates key principles that can help 
guide other state efforts to develop measures of ELO quality. 

►► Target both sides of the quality equation. APT measures 
ELO adherence to program standards and SAYO measures 
program impacts. Using both of these types of measurement 
tools provides Massachusetts with a more complete picture of 
ELO quality, balances the strengths and weaknesses of each 
measurement tool, and supports the dual goals of program 
improvement and accountability. 

►► Create tools that adapt to different program types. Both 
SAYO and APT use a “menu” approach to assessment, 
whereby programs have some choice regarding what measures 
will apply to them. With respect to areas of academic focus, 
for example, one program may select measures on reading and 
writing skills, whereas another may select measures on science 
and mathematics. This flexibility means SAYO and APT are 
adaptable to a wide variety of ELOs. 

►► Focus on reasonable and research-based outcomes. SAYO 
enables ELOs to assess changes in youth outcomes that are 
associated with participation in high quality ELOs and likely 
to occur in a single year. For example, SAYO measures “inter- 
mediary effects” (e.g., increases in student engagement, self- 
esteem, school attendance, and work-study habits) that serve 
as precursors to improved academic achievement, which may 




develop over longer periods.^^ APT, though it does not directly 
measure participant outcomes, measures program adherence 
to research-based best practices. In the Massachusetts After- 
school Research Study, program quality as measured by APT 
was linked to improved program outcomes on SAYO. 

►► Ensure that measures are statistically sound. Both SAYO 
and APT went through extensive field-testing to ensure their 
validity and reliability for measuring change in elementary 
and middle school students. Sets of survey questions were 
tested to ensure they could be incorporated into scales that 
effectively captured the outcomes being measured. 

►► Make use of multiple measures and sources of data. SAYO 
collects data from regular education teachers, participants, and 
ELO staff on student academic achievement, homework com- 
pletion, behavior, initiative, relationship development, program 
experiences, and engagement in learning, among other out- 
comes. Using multiple measures and multiple sources of data 
(e.g., participants, providers, parents, teachers, and trained 
observers) provides a more complete and balanced representa- 
tion of program quality. 

The National Institute on Out-of-School Time is working to 
expand the reach of SAYO and APT by offering them as part of 
an Afterschool Program Assessment System (APAS).^^ Georgia 
is now piloting APAS in 15 sites. 
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Action 5: Provide Incentives to Improve ELO Quality 



S tates should provide incentives for programs to reach 
for higher levels of program quality. Increasingly, 
states are using quality rating systems and mechanisms 
that tie funding to program quality to spur ELOs to 
achieve excellence. 

A Quality Rating System (QRS) rates ELOs according to 
whether they meet particular quality benchmarks or stan- 
dards in an easy-to-understand manner (e.g., on a scale of 
one to five stars). These rating systems generally build on 
minimum requirements set out in state statute or regula- 
tions so that meeting these requirements means a program 
has met the first benchmark in the QRS (see Use Regula- 
tions to Set a Floor for Program Quality). A top rating in a 
QRS could be used to demonstrate that a program has met 
state ELO quality standards. At least 13 states have a QRS 
that applies to school-age youth in afterschool programs.^^ 
For example, Keystone STARS, the Pennsylvania QRS, 
groups programs into four levels (one, two, three, and four 
stars) depending on the research-based performance bench- 
marks that programs meet. The benchmarks relate to staff 
qualifications and professional development, curricula and 
assessment, partnerships with family and community, and 
leadership and management. Pennsylvania, Missouri, 
and Ohio have been recognized as leaders in ensuring that 
their quality rating systems work well for ELOs, given that 
many of these systems were originally developed for early 
childhood care and education. 



By rating ELOs on their quality and 
publicizing these ratings, states can pro- 
vide consumers with solid information 
to inform their decisions. 



To ensure that relatively high quality programs are more 
likely to be funded than relatively low quality programs, 
states can structure state ELO funding streams as com- 
petitive grant programs. For example, “Lottery for Edu- 
cation: Afterschool Programs” is an ELO grant initiative 
supported by funds from unclaimed lottery winnings in 
Tennessee. Funded at $12.5 million in 2006, the pro- 
gram supports quality with a rigorous application 
process that requires a needs assessment, detailed project 
design information, data on performance measures, evi- 
dence of effective partnerships, and a well-formulated 
budget. For its first round of grants, the state funded 20 
of the most promising projects out of 133 applications. 
In later rounds, high quality projects were granted con- 
tinuation funds, even as new projects were awarded. 

Beyond state-specific programs, states have considerable 
responsibility in administering federal 21st CCLC grants, 
which they can take advantage of to boost the quality of 
grantees. States manage 21st CCLC grant competitions, 
award grants to eligible entities, monitor programs to 
ensure that statutory requirements are being met, and 
develop performance indicators to measure and evaluate 
programs. In addition to these requirements, states are 
given some discretion to determine key aspects of the 21st 
CCLC program, including the amount and duration of 
grant awards, competitive priorities in state competitions 
beyond those required by federal law, and availability of 
technical assistance for eligible applicants. The ability to 
structure the state competitive awards process for 2 1 st 
CCLC gives states the opportunity to maximize the 
quality of programs that are ultimately funded. 



By rating ELOs on their quality and publicizing these 
ratings, states can provide consumers with solid informa- 
tion to inform their decisions. Rating and reporting also 
provide programs with a compelling market-based rea- 
son to raise their quality level. 



The Quality Imperative: 

A State Guide to Achieving the Promise of Extended Learning Opportunities 



The ability to structure the state 
competitive awards process for 21st 
CCLC gives states the opportunity to 
maximize the quality of programs that 
are ultimately funded. 

States can provide programs with another powerful in- 
centive to raise their quality levels by tying higher levels 
of ELO program funding to improved quality. States 
can, for instance, use tiered reimbursement systems to 
provide higher rates of reimbursements to higher-quality 
afterschool programs. In Pennsylvania, for example, pro- 
grams that serve low-income children through the states 
subsidized child care program can earn higher reimburse- 
ment rates by scoring higher ratings on the Keystone 
STARS Quality Rating and Improvement System; as of 



fiscal 2008-09, compared with one-star programs, two- 
star programs receive an extra $0.50 per day for each 
full-time enrolled child, three-star programs receive an 
extra $1.50, and four-star programs receive an extra 
$2.00.^^ Pennsylvania also provides additional financial 
awards to eligible programs to help them improve pro- 
gram quality and to raise education and retention rates 
of key program staff. For more information on Keystone 
STARS, please see: http://www.pakeys.org/stars. 

At least 30 states have a tiered reimbursement system, 
and all but a few of them include school-age care pro- 
grams in their system. However, these systems typically 
apply only to programs funded with child care dollars, 
not to other school- or community-based ELOs. More- 
over, many tiered reimbursement systems focus on struc- 
tural issues, such as staff qualifications, and fail to take 
student outcomes into account. 



Use Regulations to Set a Floor for Program Quality 



States should set and enforce basic requirements that all ELOs 
must meet. In the child care field, these basic requirements are in- 
corporated into licensing regulations, typically focused on ensuring 
participant health and safety. All states have child care licensing 
regulations, covering areas such as physical environment, child-staff 
ratios, staff qualifications, and basic procedural requirements.^^ A 
number of states have made changes to their licensing regulations, 
which originally may have been crafted to support early childhood 
care, to make them relevant for ELOs. For example. South 
Dakota updated its child care regulations to recognize school 
building codes and construction rules so that school-based ELOs 
would not need to submit floor plans to child care agencies before 
receiving a license.^^ Massachusetts, Oklahoma, and Pennsylva- 
nia allow program administrators of licensed school-age care facili- 
ties to hold degrees in elementary or secondary education, or other 
relevant disciplines, as opposed to early childhood education. 

States should establish minimum quality regulations for other 
federal and state ELO programs. Supplemental Educational 
Services (SES), for example, provides tutoring services to eligible 
students in Title I schools that have not made adequate yearly 
progress for three years. States are responsible for determining 
what providers are eligible to provide SES, and they can struc- 
ture and refine the application process to help ensure that only 
providers meeting established criteria related to quality services 



are approved. California’s Afterschool Education and Safety 
program provides an example of minimum quality regulations 
for a state ELO funding stream. Under that program: 

►► All staff members who directly supervise participants must 
meet the minimum qualifications, hiring requirements, and 
procedures for an instructional aide in the school district; 

►► The maximum participant-to-staff member ratio is 20 to 1 ; 

►► Participants must receive a daily nutritious snack; and 

►► Programs must operate every regular school day until 6:00 
p.m. and a minimum of 15 hours per week. 



Meeting basic requirements is necessary 
for ELOs to be effective, but by no 
means sufficient. 



Meeting basic requirements is necessary for ELOs to be effec- 
tive, but by no means sufficient. Consequently, states must de- 
velop and implement policies that provide incentives for 
improved quality. 
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As states and ELOs increase their abilities to measure the 
impact of ELOs, states should consider ways to reward 
programs that show positive results. For example, states 
could require that programs demonstrate positive effects 
on student outcomes in order to reach the highest levels 
of quality rating and tiered reimbursement systems. Sim- 
ilarly, states could require ELOs receiving funding via a 
competitive grant program to submit evidence of their 



direct impact on children, families, and communities. 
States could use this information to reward successful 
programs with, for example, continuation grants. They 
could also provide struggling programs with additional 
supports to improve quality and withdraw certification 
or funding from consistently low quality programs. 



Action 6: Support a Strong ELO Workforce 



E LO quality rests on the strength of the individuals 
working in the programs. ELOs are typically staffed 
by some mix of teachers, youth workers, hourly staff, 
and volunteers. Because of the part-time, often volun- 
tary, nature of much of the ELO workforce, fostering 
quality is a formidable task. To support a strong ELO 
workforce, states can help recruit talented individuals 
into the field, provide professional development to staff 
members, and retain strong staff members. 

Use Innovative Approaches to 
Recruit Talented Individuals 

States can help recruit talented frontline staff into the 
ELO field by making ELO staffing a part of state commu- 
nity service and service learning programs. In Massachu- 
setts, for example. Governor Deval Patrick worked with 
legislative leaders to create the Commonwealth Corps, a 
group of volunteers who earn a modest stipend and com- 
mit to a year of full-time or part-time community service 
activities, including service to afterschool and mentorship 
programs. In Maryland, all students must participate in 
community service as a high school graduation require- 
ment. Several Maryland school districts allow older stu- 
dents to earn their community service credits serving as 
tutors for younger students in afterschool programs.^^ In 
all states, governor-appointed state service commissions 
manage, monitor, evaluate, and provide corporation fund- 
ing to state AmeriCorps programs.^^ These commissions 
can ensure that high quality ELOs are included as organi- 
zations eligible to receive AmeriCorps grants, and, more 
broadly, can encourage volunteering in support of high 
quality ELOs in their state. 



States can also help ELOs farm two- and four-year col- 
leges and universities for talented undergraduates who are 
looking for part-time nonprofessional employment, a rich 
work experience, and the opportunity to have a meaning- 
ful impact on the lives of children and youth. For exam- 
ple, states can support the inclusion of high quality ELOs 
as eligible employers in state and federal work-study pro- 
grams and in teaching fellows programs. For example, 
college engineering students could be encouraged to work 
part-time in ELOs that help high school students learn 
the fundamentals of mechanical and electrical engineer- 
ing through robotics competitions. Imagine college stu- 
dents on a teaching career path who earn both money 
and real-world teaching experience tutoring students at 
ELOs. This is happening in Fresno County, California, 
where local college students aspiring to be teachers are 
recruited, trained, and placed in afterschool programs for 
a minimum of 1 5 hours per week as teaching fellows. The 
college students, who must maintain a 3.0 grade point 
average, receive a living stipend, ongoing training, and 
college credit for related coursework.^^ 

States can support the development of ELO program lead- 
ership through partnerships with higher education institu- 
tions. In Massachusetts, for example, Lesley University 
partnered with Citizen Schools, a local program provider, 
to develop a master s program that grants degrees in educa- 
tion with a specialization in out-of-school time. Through 
higher education governance structures, states can ensure 
that sufficient capacity exists to train leaders for the grow- 
ing ELO field. To make a career as a leader in the ELO 
field more attractive, states can extend programs tradition- 
ally aimed at school teachers to cover ELO program coor- 
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dinators. For example, states can provide scholarships, col- 
lege loan forgiveness, and other incentives (e.g., new home 
buying tax credits) to individuals who pursue a career in 
ELO management and remain in that profession, serving 
high-need populations, for a given amount of time. 

Provide Professional Development 
Opportunities 

To help ELOs reach higher quality levels, states can sup- 
port the availability of regular professional development 
opportunities for program staff. Building on insights 
from early childhood care and education. Building Pro- 
fessional Development Systems for the After school Field, an 
Afterschool Investments Project report published by the 
U.S. Department of Health and Human Services’ Child 
Care Bureau, provides a road map for how states can 
support ELO professional development:^^ 

►► States can provide funding and develop incentives to 
support ELO staff professional development. For 

example, seven states have incorporated afterschool 
providers into their versions of the TEACH Early 
Childhood Project, which provides scholarships to child 
care providers and rewards staff members who boost 
their education with increased wages or a stipend. States 
can also incorporate programs’ professional develop- 
ment activities as an indicator in quality rating and 
tiered reimbursement systems. Most funding for profes- 
sional development system building comes from Child 
Care and Development Fund dollars. 

►► To create a framework for professional development 
activities, states can establish core knowledge and 
competencies that delineate what afterschool workers 
should know and be able to do. For example, Kansas 
and Missouri worked together to develop the Kansas 
and Missouri Core Competencies for Youth Develop- 
ment Professionals, which covers knowledge regarding 
how children and youth learn and develop, how to es- 
tablish appropriate learning environments and curricula, 
and how to establish and maintain a healthy and safe 
environment for participants. Minnesota has also devel- 



oped core competencies, along with a career lattice for 
afterschool workers. States can use “career lattices” to 
link providers’ levels of experience, education, and train- 
ing to various positions and compensation levels. 

►► States can also create credentials that provide a 
means for afterschool workers to gain recognition 
for their professional development training. As a 

case in point, Connecticut has created a Credential in 
After School Education for individuals who specialize 
in the care and education of children, ages 5 to 15, 
during out-of-school time. To earn the credential, af- 
terschool workers must pass four online courses of- 
fered by a state college, document that they have 
received 240 hours of experience in afterschool educa- 
tion, submit a professional resource file, and complete 
a field experience course. 

►► To make it easier for ELO workers to access train- 
ings, states can provide a one-stop shop for access- 
ing information on professional development 
offerings, and they can make use of regional technical 
assistance structures to enhance outreach. Ohio’s pro- 
fessional development registry is a good example of a 
statewide ELO training clearinghouse, which also al- 
lows trainees to track their progress as they participate 
in trainings. North Carolina provides an example of a 
state that uses regional training specialists who design 
and deliver trainings keyed to local needs and collabo- 
rate with public school systems to provide profes- 
sional development for individuals working in 
school-based programs. 

►► To guarantee the quality of the trainings, states can 
certify instructors. In Pennsylvania, for example, to be 
listed in its catalog of certified instructors, individuals 
must receive training on the components of the state’s 
professional development system, techniques to effec- 
tively foster adult learning, and methods for aligning 
courses with the state’s core body of knowledge for 
school age care professionals.^^ 
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Given the overlap between professional development for 
afterschool workers and regular school-day staff, states 
can offer guidance for programs on working closely with 
schools and school districts. States can also create an inte- 
grated system of professional development that addresses 
both ELO and school-day staff.^^ In supporting profes- 
sional development for ELO staff, states should take into 
account lessons learned from research on professional 
development for regular school-day staff. 



In supporting professional development 
for ELO staff, states should take into 
account lessons learned from research 
on professional development for regular 
school-day staff. 

Furthermore, states can target program areas identified by 
evaluation as weak or needing improvement and partner 
with other organizations and agencies to increase the state s 
capacity to support effective professional development.^^ 



Action 7: Connect Students with High Quality ELOs 



S tates cannot reap the full benefits of high quality 
ELOs if programs are unavailable to those who need 
them. Gaps in availability persist, particularly for low- 
income and minority families that have a harder time 
accessing high quality programs in their communities. 
Nationally, more than 14 million children take care of 
themselves after the school day ends, including almost 
4 million middle school students. 



Nationally, more than 14 million 
children take care of themselves after 
the school day ends, including almost 
4 million middle school students. 



Low-income and minority students face particular obsta- 
cles, as about two-thirds of their parents report finding it 
difficult to find high quality ELOs.^^ To connect students 
with high quality ELOs, states can analyze ELO supply 
and demand, target investments in high quality ELOs to 
underserved populations, and inform parents and stu- 
dents of high quality ELOs. 



Perform an ELO Supply and 
Demand Analysis 

States can conduct an ELO supply and demand analysis 
to determine which students are participating in high 
quality ELOs, which students are participating in lower 
quality ELOs, and which students are doing without. 
States typically have access to information on ELO sup- 
ply from the state education agency, which compiles 
information on 2 1 st Gentury Gommunity Learning 
Genters and other ELO programs, and state child care 
offices or child care resource and referral agencies, which 
compile data on licensed child care centers that serve 
school-age children. However, this information pro- 
vides an incomplete picture of ELO supply, given that 
many programs are funded with private or local dollars. 
Similarly, although states have some information on de- 
mand thanks to program waiting lists and routine collec- 
tions of demographic data, this data may be insufficient. 
Because of these issues, states may collect additional data 
via provider and parent surveys. Montana, for example, 
compiled data from its state education agency, the Annie 
E. Gasey Foundations Kids Gount publication, a tele- 
phone survey of licensed school-age care centers, and a 
telephone survey of state-level youth organizations and 
local informants. Using data from its analysis, Mon- 
tana generated county- by- county maps that show the 
percentage of school-age children with working parents 
participating in ELOs.^®^ 
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For supply and demand analyses to be most beneficial, 
however, they should incorporate information on the 
quality of the ELO supply. By linking a supply and de- 
mand analysis with a quality rating system or other method 
of measuring afterschool quality, states can determine 
unmet need and target resources to improve program qual- 
ity or increase program access accordingly. 

Few, if any, states have incorporated comprehensive meas- 
ures of quality into their ELO supply and demand studies. 
In Pennsylvania, however, the statewide afterschool net- 
work has had preliminary discussions on creating a state 
ELO data system that would track information on ELO 
availability, use, and quality and link to the state s broader 
education data system. By analyzing the demand for ELOs 
and the supply of high quality ELOs in a periodic or on- 
going manner, such a state system would provide data to 
support smart investments in ELOs. 

Target Investments in High quality ELOs 
to Underserved Populations 

Given budget limitations, most states are targeting under- 
served populations so that scarce resources can do the most 
good. Some states are approaching this issue by supporting 
ELOs for students, schools, and districts that are facing 
academic challenges. For example, Connecticut’s Priority 
Extended School Hours Grant awards funds for ELOs — 
$3 million in 2006-07 — to districts with poor student per- 
formance on state exams, high levels of welfare receipt, or 
large populations. Illinois’ Summer Bridges program, a 
summer reading and writing program for students in 
prekindergarten through grade six-budgeted at $22 million 
in 2007 — directs support to students who do not meet 
state reading standards and who attend school in a district 
where at least one school has 50 percent or more of its stu- 
dents failing to meet these standards. 

Beyond providing funding specifically for ELOs, states 
can include high quality ELOs as allowable expenses 
under more flexible funding streams. Gonsider New 
fork’s Gontracts for Excellence, as an example. Begin- 
ning in the 2007-08 school year, school districts that had 
underperforming schools and that received a 10 percent or 
$ 1 5 million increase in state funding had to develop spe- 
cific plans to use these new funds to improve student 



Beyond providing funding specifically for 
ELOs, states can include high quality 
ELOs as allowable expenses under more 
flexible funding streams. 



performance. These plans, embodied in Gontracts for 
Excellence, had to focus on reforms proven to increase stu- 
dent achievement, such as academic ELOs, smaller class 
sizes, or prekindergarten. Similarly, Oregon provided 
school districts with $260 million in new funding for the 
2007-09 biennium to be spent on activities to improve stu- 
dent achievement, including ELOs, teacher professional 
development, and literacy programs. The North Car- 
olina Disadvantaged Student Supplemental Fund is an- 
other flexible funding stream that can be used to support 
ELOs. This fund, which allotted $69 million to local edu- 
cation agencies in 2007-08, provides funding according to 
the number of district students who are from low-income 
families, live in single-parent households, or have at least 
one parent with less than a high school diploma. Incor- 
porating high quality ELOs as allowable expenses under 
flexible funding streams highlights the positive impact 
these initiatives can have and makes them a more integral 
part of state and local education systems. 

States can also tap funding from Temporary Assistance for 
Needy Families (TANF) to provide afterschool program- 
ming to students of eligible families. Although funds avail- 
able for afterschool programming vary based on general 
demand (i.e., applications) for TANF funds, the funding 
requirements for TANF help ensure that children with the 
greatest need are being served. TANF funds can be trans- 
ferred to a state’s Ghild Gate and Development Fund or 
used to directly fund programs. Georgia, for instance, has 
allocated $14 million in TANF funds to support ELOs, 
including both afterschool and summer learning pro- 
grams, focused on youth development. Ohio has allocated 
$10 million per year in TANF money for afterschool for 
2008 and 2009. Louisiana uses TANF funds to support 
ELOs, many of which are operated by community- and 
faith-based organizations. The state department of educa- 
tion administers Louisiana’s TANF funds for ELOs, which 
were allocated at $12.5 million for fiscal 2007. 
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Several states are focusing on ELOs for older youth because 
this population is typically underserved in the field. The 
percentage of children taking care of themselves at least 
once per week after school jumps from 2 percent in grades 
K-2, to 7 percent in grades 3-5, and 27 percent in grades 
q£ structured activities for many middle 
school students may help explain why U.S. students’ aca- 
demic performance tends to fall off in the middle grades. 
Consequently, states are investing in ELOs for youth in 
middle and high school. For example, Colorado’s dropout 
prevention activity grant fund supports before-school and 
afterschool arts-based and vocational programs for students 
in middle and high school. Middle and high schools that 
received a “low” or “unsatisfactory” rating in the state’s sys- 
tem are eligible for the program. Supported with $400,000 
in state funding in 2008, the Kansas Middle School After- 
school Activity Advancement Grant funds afterschool and 
summer programs that provide middle school youth with 
age-appropriate physical activity, academic enhancement, 
and career and higher learning opportunities. California 
is helping to lead the charge in support of ELOs for high 
school students with its After School Safety and Enrich- 
ment for Teens (ASSETs) Program. Having invested sub- 
stantial state funding in afterschool for grades K-8, the state 
education agency directed $43 million in 21st CCLC fund- 
ing to ASSETS for the 2007-08 school year.^^^ A recent 
evaluation of ASSETs found that program participants 
passed the English and mathematics portions of the state’s 
high school exit exam at a higher rate than similar students 
not involved in the program. 

Inform Parents and Students of 
High Quality ELOs 

Several states are focusing on ELOs for 
older youth because this population is 
typically underserved in the field. 



To increase the demand for quality ELOs and to address 
information gaps, states can provide parents and students 
with information on the features and benefits of high qual- 
ity ELOs and on the availability of high quality ELOs in 
their communities. State Child Care Resource and Referral 
(CCR&R) networks, present in 38 states, are one means of 
accomplishing these goals. The Minnesota CCR&R 



network, for example, helps match families with high qual- 
ity ELOs and informs state residents of the importance of 
high quality ELOs through a variety of outreach and com- 
munication mechanisms.^ There are, of course, other ve- 
hicles for informing parents and students of available 
ELOs. New Jersey has developed a free and easy-to-use on- 
line directory of ELOs statewide, with $450,000 in fund- 
ing from a local energy company and leadership from a 
statewide public-private ELO partnership. The directory, 
available in both English and Spanish at www.njafter- 
school.org, allows families to easily find ELOs and 
providers to easily add and edit listings of their programs. 
Parents and students can search among the 2,000 listed 
programs by location, program activities, ages served, 
schedule, and cost. Efforts like these are critical to ensure 
that information about program opportunities reaches all 
families, including those in low-income communities, and 
that families understand the benefits of high quality ELOs. 

States can also support action at the local level that con- 
nects students with community supports to facilitate stu- 
dent success. For example, since 1990, Kentucky has 
funded Family Resource and Youth Services Centers to 
help families and children find local solutions to prob- 
lems that interfere with student learning. ELOs are a 
core component of the centers for children ages 4-12, as 
is career exploration and development for older youth. 
The centers may provide ELOs directly, contract with 
local providers, or serve as a liaison to community pro- 
grams. Funded at $55.6 million in fiscal 2008, the 820 
centers across the state serve 1 , 1 66 schools, with an 
enrollment of 612,741 students. In Georgia, Governor 
Sonny Perdue has championed an initiative, funded at 
$53 million in fiscal 2009, that puts “graduation coaches” 
into middle and high schools. The graduation coaches 
identify students at risk of dropping out, work with these 
students to create a plan for graduating successfully, and 
connect the students with community resources, such as 
ELOs, that can help them meet their academic and life 
goals. To support their efforts to connect youth with 
community resources, graduation coaches partner with 
volunteer business leaders across the state, called “com- 
munity coaches.” Although not focused on ELOs alone, 
Georgia’s use of graduation and community coaches pro- 
vides a strong example of how states can help students get 
access to the supports they need for success. 
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tate leaders recognize that afterschool, summer learn- 
ing programs, and other ELOs hold great promise 
for addressing a range of state policy challenges. ELOs 
can help prepare youth for college and career, support 
working families, and keep communities strong. It is 
because of this promise that states have ramped up 
investment in ELOs to boost the availability of these 
programs, particularly among low income and minority 
youth who have historically lacked equal access to such 
opportunities. 

Recent research has confirmed the immense potential of 
ELOs while simultaneously highlighting the need to sup- 
port ELO quality. Participation in high quality ELOs is 
linked to substantial improvements in academic achieve- 
ment, school attendance, student engagement, work- 
study habits, and social and emotional development. 
Attending a low quality program, however, can have neg- 
ative effects. Consequently, it is imperative for state poli- 
cymakers to invest in ELO quality. 

Fortunately, it is now clear what makes an effective ELO. 
Also, states have begun to develop promising policies and 
practices that support ELO quality and offer lessons 
learned. For example, states can look to North Carolina for 
ELO program standards, Massachusetts for measures of 
ELO quality, Pennsylvania for a strong quality rating sys- 
tem, Kansas and Missouri for core competencies for pro- 
fessional development, and California for methods of 
bringing high quality ELOs to scale. 



With this knowledge, governors and chief state school 
officers can establish a state ELO quality system. To cre- 
ate a comprehensive system, state leaders can convene an 
ELO quality team and empower that team to set state 
goals for ELOs along with program standards and meas- 
ures of ELO quality. Furthermore, governors and chief 
state school officers can work with their ELO quality 
teams to champion policies that support a strong ELO 
workforce, provide incentives to improve ELO quality, 
and connect students with high quality ELOs. 

By ratcheting up disparate, oftentimes disconnected 
ELO quality efforts, state leaders who establish well- 
functioning state ELO quality systems can expect sub- 
stantially higher returns on their investments in ELOs. 
More profoundly, by making good on the promise of 
ELOs, chief state school officers and governors can help 
ensure the success of children, families, and communities 
in their states. 
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^ A broader definition of ELO, which includes extended day and year 
initiatives, distance learning, and early childhood education initia- 
tives, is included in the Council’s policy statement on extended 
learning opportunities. However, these types of initiatives are 
beyond the scope of this report which focuses solely on programs 
serving school-age youth and occurring outside the context of the 
regular school day. 

^ Miller, K. & Snow, D. (2004). Noteworthy perspectives: Out-of-school 
time programs for at-risk students. Aurora, CO: Mid-Continent Re- 
search for Education and Learning. See also, Mahoney, J.L., Lord, H. 
& Carryl, E. (2005). An ecological analysis of after-school program 
participation and the development of academic performance and mo- 
tivational attributes for disadvantaged children. Child Development, 
76(4), 811-825. See also, Vandell, D.L., Reisner, E.R., & Pierce, 

K.M. (2007). Outcomes linked to high quality afterschool programs: 
Longitudinal findings from the study of promising afterschool programs. 
Retrieved September 8, 2008, from http://www.afterschool 
resources. org/kernel/images/*NDLHIllPromisingPracticesReportfi- 
nal.pdf See also. Little, P.M.D. & Harris, E. (2003). A Review of 
Out-of-School Time Program Quasi-Experimental and Experimental 
Evaluation Results. Out-of-School Time Evaluation Snapshot, No. 1. 
Cambridge, MA: Harvard Family Research Project. See also, Huang, 
D., Gribbons, B., Kim, K.S., & Lee, C. (2000). A decade of results: 

The impact of the LAs BEST after school enrichment program on subse- 
quent student achievement and performance. Los Angeles, CA: UCLA 
Center for the Study of Evaluation. Retrieved August 21, 2008, from 
http://www.lasbest.org/resourcecenter/uclaeval.pdf See also, Vandell, 
D.L., Reisner, E.R., Brown, B.B., Dadisman, K., Pierce, K.M., Lee, 
D., & Pechman, E.M. (2005). The study of promising programs: Exam- 
ination of intermediate outcomes in year 2. Madison, Wisconsin: Uni- 
versity of Wisconsin Center for Education Research. See also, Durlak, 
J.A., &Weissberg, R.P. (2007). The impact of after-school programs that 
promote personal and social skills. Chicago, IE: Collaborative for Aca- 
demic, Social and Emotional Learning. See also, Warren, C., Feist, 

M., & Nevarez, N. (2002). A place to grow: Evaluation of the New York 
City Beacons. New York, NY: Academy for Educational Development. 
See also, Grossman, J.B., Price, M.L., Fellerath, V, Jucovy, L.Z., Kot- 
loff, L.J., Raley, R., & Walker, K.E. (2002). Multiple choices after- 
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